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Abstract. We establish necessary and sufficient conditions for consistency of 
estimators of mixing distribution in linear latent structure analysis. 



1. Introduction 

Linear latent structure (LLS) analysis is aimed to derive properties of population 
as whole and properties of individuals from a large number of categorical measure- 
m ents made on each ind ividual in a sample. An exposition of LLS analysis is given 
in lKovtun etaH l)2005b|) . 

LLS analysis searches for representation of the observed joint distribution of 
random variables (representing measurements) as a mixture of independent distri- 
butions, i.e. distributions, in which random variables are mutually independent. 
Such approach is common for all branches of latent structure analysis. The specific 
LLS assumption is that the mixing distribution is supported by a low-dimensional 
linear subspace of the space of independent distributions. 

When dimensionality of the supporting subspace is sufficiently smaller than the 
number of random variables and under some regularity conditions, a set of low- 
order moments (of order up to number of measurements) of mixing distribution 
is identifiable. Increasing the size of the sample does not increase the number of 
identifiable moments — only precision of their estimates incre ases. Thus, the mixing 
distribution in LLS analysis is only partially identifiable. In lKovtun et all (|2005b) 
we suggested an estimator of the mixing distribution. This estimator, however, 
does not converge to the true mixing distribution when the sample size tends to 
infinity. The natural question arise: what is the value of such partially identifiable 
model and in what sense the suggested estimator is useful? 

In this article, we give one possible answer to this question. Assume that one has 
an infinite sequence of possible measurements; a particular survey uses finite number 
of measurements from this sequence. Then a sequence of estimates of mixing distri- 
bution obtained in a sequence of surveys with increasing number of measurements 
converges to the true mixing distribution. The necessary and sufficient condition 
for this convergence is pairwise orthogonality of independent distributions being 
mixed. 
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The crucial part of the proof is the equivalence between pairwise orthogonality 
of independent distribution and orthogonality in aggregate (see lemma 15.2411 . This 
fact is of interest on its own; we devote a special discussion to its consequences with 
respect to structure of tail er-algebras and the strong law of large numbers. 

The fact that pairwise orthogonality of a family of measures being mixed is 
necessary and sufficient for equality G = E(G| : 5 r00 ) u-a.s. (see definition s below) 
was proved by A. Yashin for the case of countable families of measures (see lYashfnl 
Il98a Il98rt where this fact was used to prove the strong consistency of statistical 
estimates in the adaptive control schemes). A generalization of this fact to the case 
of families of measures of cardinality of continuum, proved by M. Kovtun, provides 
a technical basis for the results of this paper. 

The rest of the article is organized as follows. We start with extensive prelim- 
inaries in order to make the article self-sufficient for wider audience. Section 
provides all necessary definitions and establishes basic facts. Section 0] contains a 
proof of the main theorem. Proofs of technical facts are given in section [3] The 
article is concluded by discussion of connections between our proofs and structure 
of tail cr-algebras and the strong law of large numbers. 



2. Preliminaries 

2.1. Basic notions and notation. For a topological space O, 3(0) denotes its 
Borel er-algebra (i.e., a-algebra, generated by open subsets of O). 

Let fi be a Borel probabilistic measure on a topological space O. We say that fi 
is carried by a set A £ 23(0), if fi(A) = 1. A measure fi is supported by a set A, if 
fi is carried by A and A is closed. A closed set S is a support of measure fi, if S 
supports \x and S — C\{A \ A supports fi}. The support of measure fi is denoted by 
supp(/x). Every measure on topological space with countable base (in particular, 
on separable metrizable space) has a support. 

Note that a set that carries a measure fi can be "significantly smaller" than its 
support. For example, let {r n }^ =1 be any enumeration of rational numbers of the 
interval [0, 1], and let fi r be a counting measure defined by fi r (r n ) = 2~ n . Then the 
set of rationals carries fi r , but the support of fi r is the whole interval [0, 1]. 

Two measures fi and fjf are said to be orthogonal (or singular), denoted fi _L //, if 
there exists a set A £ 23(0) such that A carries fi and Cl\A carries fi' . For example, 
fi r is orthogonal to Lebesgue measure A on [0, 1], as fi r is carried by rationals and 
A is carried by irrationals. 

We write fi <C fi' if H is absolutely continuous with respect to fi' , and fi ~ ii! if 
fi is equivalent to // (i.e., fi <C li! and Li! <C n). 

For a measurable space (fi,^), a set B £ is called an atom of a -algebra 3, if 
for every B' £ J, B' C B, cither B' = B or B' = 0. For a space with measure 
(O, 3", n), a set B £ IF is called an atom of measure fi, if fi(B) > and for every 
B' £ 1, B' C B, either fi(B') = li{B) or li(B') = 0. For example, atoms of 23([0, 1]) 
are one-points sets, while the Lebesgue measure A on ([0, 1], 23([0, 1])) has no atoms. 
We say that a measure fi is continuous if it has no atoms. 

We use y(O) to denote a space of probabilistic Borel measures on a topological 
space O. We always consider y(O) with topology of weak convergence. 

In a topological space, [B] denotes the closure of a set B. In a metric space, U £ x 
denotes e-neighborhood of point x, and U e B denotes e-neighborhood of set B. Ib(-) 
denotes a characteristic function of a set B. For a set B with fi(B) > 0, fi{-\B) 
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denotes a probabilistic measure conditional on B (i.e., ii(-\B)(B') = fi(B'\B) = 
n(B'nB)/fi(B)). B n I B abbreviates the statement "Bi 3B 2 D... and f|„ B n = 
B." B n JJ B abbreviates the statement "B n J, B and Ve>03mVn>m: B n C 
U £ B." 8 X denotes a measure concentrated at a single point x. 

We shall use the following properties of Borel measures on a metric spaces: 

(1) x g supp(/i) <^> Ve > : ii(U e x) > 

(2) For every x £ supp(/i) there exists a sequence of sets {B n } n such that 
M (S„) > and B n Jj {x} 

(3) // <C jtx => supp(//) C supp(/i) 

(4) If > 0, then fi(-\B) < /i and supp(^(-|S)) C supp(/z) 

(5) If £?„ JJ. {x} and /x n is carried by B n , then /i n — ► i5 x 

2.2. Lebesgue spaces. The notion of Lebes gue space was introdu ced and i ts ma in 
properties were established in iRokhlinl ljl94fl|) (English translation- lRohlinl (|l952h ). 

Let (O, /z) be a space with measure. In this subsection, all spaces with measure 
are assumed to be complete, i.e. B S /x(£?) — 0, and B' C B imply B' S 3\ We 
also assume that /i(f2) = 1. 

A countable system of measurable sets 25 = {1^}°^ is called a basis of (CI, /x), 
if it possesses the following two properties: 

(£) VB e J 3Ce <r(<5) : B C C A li(B) = p(C) 
(UK) Vw^'el! BrfwGl.Aw'^I.jvfw'el.Aw^ T 2 ) 

A space with measure is called separable, if it has a basis. 

If is a second-countable To topological space (in particular, if f2 is a separa- 
ble metric space) and /j is a measure defined on its Borel cr-algebra 23(0), then 
(fi, 23^(0), £i), where 23 M (f2) is Lebesgue completion of Borel a-algebra with respect 
to measure /x and /x is continuation of /x onto 23^ (f2), is a separable space with 
measure, and topology base serves as a basis in the above sense. 

Let (fi, CF, /i) be a separable space with measure, and let (5 = be its basis. 
Consider intersections 

oo 

(1) C\ Ti, where either Tj = F$ or Tj = fl \ Ti 

t=i 

Due to property (9Jt), every such intersection cannot contain more than one 
point. Furthermore, every one-point set is representable in form JQ| (to obtain a 
set consisting of u>, take Ti — Ti if u> S T^, and Tj = O \ otherwise). 

A separable space with measure (f2,!F, /x) is said to be complete with respect to 
basis & = {r^ji, if every intersection Q is nonempty (and thus, consists of one 
point). A separable space with measure (0, 5F, /x) is said to be complete (mod 0) 
wifft respect to basis (5 = {r^, if it is isomorphic (mod 0) to a space with mea- 
sure (O', 5F', //), which is complete with respect to its basis ©' = and this 
isomorphism transfor ms base p to basis & . 

Important fact is I Rokhlinl . Il949t FRohlinL 1952L §2, n°2): if a separable space 



with measure is complete (mod 0) with respect to some basis, then it is complete 
(mod 0) with respect to every other basis. 

A separable space with measure is called Lebesgue space, if it is complete (mod 0) 
with respect to its bases. 

It happens that many important spaces with measures are Lebesgue spaces. In 
particular, if fl is a separable (in topological sense) complete metric space and fx 
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is a m easure defined on its Borel c r-algebra "B(fl), then (Cl, H^Q), p) is Lebesgue 
snare llRokhlinl . Il 949t l^nhlinL ll Q52L §2, n°7). 

2.3. Metric structures. Let (f2,£F, fi) be a space with measure. Then J is a 
Boolean algebra, and thus a ring (with respect to additive operation A, symmetric 
difference, and multiplicative operation n). A family of /i- negligible sets 

I,, = {B e 3 | n{B) = 0} 

is a cr-ideal of 3 r . 

The quotient algebra ^/i^ is called a me tric structure of th e space (ft, 5", fj,). The 
Boolean algebra &/la is always complete 1 VladimirovL 120021 Chapter 2). If is 



Lebesgue completion of cr-algebra J and /2 is the corresponding continuation of 
measure fJL, then ^/l„ is isomorphic (as a Boolean algebra) to ^/i . 

If B and £?' belong to the same coset with respect to ideal J M , then fi(B) — fi(B'); 
thus, /i can be considered as being defined on The quotient algebra ^/t^ can 

be made a metric space by defining a distance between two cosets a and 6 as 
p(a, b) = fi(a A b). (•%^, p) is always a complete metric space, and it is separable 
in topological sense if and only if (f2, 2F, /u) is separable in Rokhlin's sense. 

Two Lebesgue spaces are isomorphic (mod 0) as spaces with measure if and only 
if the ir metric structures are isomorphic as metric spaces faokhlinl ll94Sl iRohlirl 
Il952l §2, n°6). 

For more d etails on metric structures, see lVladimirovl l|2002h and lRokhlinl 1 1949lk 
iRohfirJ l|l952h . 

2.4. Sufficient er-algebras. In this article, we use a notion of sufficient cr-algebra, 
which is slightly more general than a notion of sufficient statistics and better servers 
for our purposes. 

Let {Pe}eee be a family of probabilistic measures defined on measurable space 
(fi, 3 r ). A cr-algebra S C 3 is said to be sufficient for family {Pe}e e e, if there exists 
a function P(oj, B), defined for all u> G £1 and Be?, such that: 

(1) For every fixed B 6 J, P{uj,B) is S-measurable (as function of oj). 

(2) For every fixed ui G CI, P(oj, B) is a probabilistic measure on (17, 5F). 

(3) For every fixed B g 5", P(u>,B) = P e (B|S)(w) P e -a.s. (i.e., P(co,B) as 
function of to is a variant of conditional probability Pe(-B|S))- 

In this terminology, a statistic T(lu) is sufficient for family {Pejeeei if cr-algebra 
cr(T), generated by random variable T(tu), is sufficient. 

3. The problem 

We consider an infinite sequence of random variables {Xj}j] variable Xj takes 
values in a finite set {1, . . . , Lj), We consider these variables as being defined 
on a probabilistic space A = rXj=i{lj • • • > Xj(a) — Xj(ai, . . .) — a,j. The 
space A, endowed with Tikhonov topology, is compact and metrizable by metric 
p(a, a') = 1/ inf{j | a,j ^ a'j}; it is complete with respect to this metric. The Borel 
cr-algebra 23 (.A) on A coincides with the cr-algebra, generated by random variables 
{Xj}j] thus, joint distributions of {Xj}j are described by Borel measures on A. 
We use J'(^l) to denote the space of all probabilistic measures on A. 

We always consider 7(A) with topology of weak convergence. Topology of 
weak convergence on 7(A) i s metriz able, ffA) is compact, and t hus separable 
l|Dunford and Schwartzl fl958t IV.6.3: lKolmogorov and Fomml . ll972l IV.3.4.4,5). 
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Among all joint distributions of {Xj}j, we distinguish independent ones, i.e. 
those distributions, in which random variables {Xj}j are mutually independent. To 
specify an independent distribution, one needs to specify only probabilities f3ji = 
P(Xj = I). Then, due to independence, one has for cylinders P{Xj ± = h A • • • A 
Xj P — lp) = Pjih ' ' ' Pjpipi an d as cylinders compose a topology base, this uniquely 
extends to r B{A). Thus, any independent distribution is uniquely described by an 
infinite-dimensional vector (3 = {fin, ■ ■ . , PiLt , Pji, ■ ■ ■ , PjLj ,■•■)■ To specify a 
distribution, such vector must satisfy conditions: 



The set of vectors satisfying © is a convex infinite-dimensional body P in IR°° . 
Let P p denote the independent distribution over A corresponding to a vector /? £ P. 

Our goal is to investigate a class of mixtures of independent distributions, i.e. 
those distributions P on A which can be represented in form: 



where \i is a probabilistic measure on P. For this definition of mixture to be 
correct, one needs to show that the mapping j3 i— > Pp(A) is measurable for every 
Borel A C A. We show a stronger fact: 

Proposition 3.1. The mapping [3 i— > Pp is continuous with respect to Tikhonov 
topology on M°° and topology of weak convergence on ?{A). 

Proof. It is sufficient to show that for every converging sequence (3 n — » [3 in P, the 
sequence Pp™ weakly converges and limP^n = Pp. For this, in tu rn, one needs to 
show that P/3»(C) -> Pp{C) for every cylinder C with Pp{dC) = llShirvaevl Eon! 
III. 1.5). In A every cylinder is open-closed set; thus, this must be shown for all 
cylinders. But for a cylinder C with base {aj 1 , . . . , aj p } one has Pp™ (C) = rj Pj' qaj , 
and required convergence follows directly from the convergence (3 n — ► /3. □ 

Corollary 3.2. for every A S 3(^1), i/ie mapping /? i— > P/3(A) ?s measurable (w.r.t. 
corresponding Borel a -algebras). 

Proof. By iDellacherie and Meve rl (I197SL III55.601. the mapping P i— > P(£?) is mea- 
surable. Thus, our mapping is measurable as a composition of two measurable 
mappings. □ 

We abbreviate the equation © to P = Mix(ju). We also use P M to denote Mix(/z). 

One of the questions that we are interested in i s conditions fo r identifiability of 
mixtures of independent distributions (in sense of lTeicheil Il960|) . i.e. under what 
conditions the mixture Mix(/x) uniquely defines the mixing measure /x. It is easy to 
show, however, that without additional restrictions any distribution on A can be 
represented as a mixture of independent ones, and (except degenerate cases) every 
distribution has infinitely many such representations. 

For a distribution /i on P, let £>(/x) denote the smallest linear subspace of M°° 
supporting fi (this subspace is a linear span of supp(/x)). We say that a mixing 
distribution /i has rank K, rank(/i) = K, if dim(£(/i)) = K. We say that a 
distribution P £ J'(-A) has rank not greater than K, rank(P) < K, if P = Mix(/i) 




(3) 
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for some measure fi with rank(/i) = K: further, rank(P) = K , if rank(P) < K and 
rank(P) £ K - 1. 

The question of identifiability can be now splitted into two subquestions: 

• For a distribution of rank K, what are conditions for identifiability of a 
K -dimensional linear subspace carrying a mixing measure? More precisely: 
what are conditions for implication 

rank(P) = K \ 

P = MixO) = Mix(//) \ => L(fi) = £(//) 

rank(/i) = rank(//) = K J 

to be true? 

• For a fixed K -dimensional subspace £ o/lR 00 , what are conditions for iden- 
tifiability of mixing measure carried by this subspace? More precisely: what 
are conditions for implication 

Mix(//) = Mix(//) 
Z(jt) = £(//) = £ 

to be true? 

We shall address these questions full detail in another article. Here, we men- 
ti on a sufficient conditio n of identifiability established (in slightly different form) 
in IKovtun et alJ l|2005b|) . Let C = (Cov^/?^, fij'i')) ji j^/ be (infinite) covariance 
matrix of the mixing measure fi. Suppose that all minors of C of size K + 1 are 
degenerate and all minors of size K are non-degenerate. Then rank(/i) = K and 
any mixing measure of rank not greater than K, which produces the same mixture, 
coincides with \x. 

When the mixing measure is identifiable, the question arise how the mixing 
measure can be estimated. The main topic of the present ar t icle is the investiga- 
tion of properties of the estimator, outlined in lKovtun et alJ 1 2005bl) . The formal 
description of this estimator is given below. 

Let us fix a i^-dimensional subspace £ of R°° such that Pq = PD £ is a (K — 1)- 
dimensional body. Let us also fix A 1 ,..., A , a set of K linearly independent 
vectors from Pq. Then every vector /3 € Pq can be uniquely represented in form 
(3 = J2k9k^ k , and ~^2 k gk = 1. Thus, we have a linear isomorphism (3 : Q — > Pq, 
where Q is a (K— l)-dimcnsional body in M. K , belonging to the hyperplane J^k 9k — 
1. For brevity, we write P g instead of Pp( g )- 

Because the mapping (3 : Q — > Pq is obviously continuous, we immediately obtain 
from proposition 

Proposition 3.3. The mapping g t— > P g is continuous with respect to standard 
Euclidean topology on Q and topology of weak convergence on CP(7l). 

Proposition 3.4. For every B 6 15(A), the mapping g t— > P g (P>) is measurable 
(w.r.t. corresponding Borel a-algebras). 

Due to the isomorphism (3 : Q — > Pq, every measure fip on Pq induces measure 
Hq on Q (by letting /j,q (B) = fip (f3(B))) and vice versa. We usually drop the index 
and write just /i, as it is obvious from the context what measure is considered. 

Thus, every probabilistic measure fx on Q generates a probabilistic measure P M 
by means of @. 

Now we develop a slightly different view on the problem. Measure [i may be 
considered as a distribution of random variable G that takes values in Q. Random 
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variables G, X\ , . . . may be considered as denned on a common probabilistic space 
Q, = Q x A; G(<?, ax, ... ) — g and Xj(g, ax, ... ) = ay. 

A distribution /i of G induces a joint distribution of G, Ax, . . . : for Borel sets 
B e B(Q) and A e B(.A), 

(4) KBx4)= / P fl (A) 

JB 

When A is a cylinder with base {a^, . . . , &j p }, one obtains 

„ p k 

u(bxa)= / nE?4»^w 

~' S ?=1 fc=X 

By the Kolmogorov's theorem, the measure v uniquely extends from sets of form 
BxCto the Borel cr-algebra of f2. 

The marginal distribution of v on A is P M , and the marginal distribution of v on 
Q is /Lt. Note that v is not a product of /i and P^. 

Let 5F" = er(Xi, . . . , X n ), an increasing sequence of c-subalgebras of 23(f2), and 
let 5"°° = a(Xi, ■ ■ ■ ) = c(Un ^™)- Consider conditional expectations E(G | J") and 
E(G | By Levi theorem, E(G | J") -> E(G | ?°°), f-a.s. and in the sense of 

L\ 

On the other hand, every element of J 71 and 3^°° has a form Q x A, A E H(A) 
(in other words, sets Q x {a} are atomic w.r.t. u-algebras 3™, 3 rco ). Thus, values 
E(G | 3 rn )(g,a), E(G | 3 ro °)(g,a) do not depend on g. This allows us to define 
functions e n : A — > Q, eoo : .A — > Q by letting e rl (a) = E(G | 3*™)(<7, a) and similarly 
for eoo- Again, e„ — > eoo P M -a.s. and in the sense of L 1 . 

Functions e„, eoo transform measure P M on .A to measures £i n , /too on Q by 
letting fi n (B) = P Al (e^ 1 (i?)) for every B e 23 (Q), and similarly for fi^,. Simple, 
but important fact is: 

Proposition 3.5. The sequence {p, n } n weakly converges to /too- 

Proof. It is sufficient to show (jShirvaevl 120041 III.1.3.V) that J (f)(g) jl n {dg) con- 
verges to J 4>(g) fj,oo(dg) for every uniformly continuous function <fi : Q — > M. By 
changing variables, one obtains 



&(g) Vn(dg) = J <j>{e n {a)) P^da) 
4>{g) Aoo(dff) = / 0(e oo (a)) P M (da) 



L 

and required convergence is equivalent to <fi o e„ — > o eoo , which follows from the 

L 1 

convergence e„ — > eoo due to the uniform continuity of <j). □ 

Functions e„ do not depend on "tails" of their arguments: if cij = a'j for j = 
l,...,n, then e„(a) = e n {a'). This allows to construct f2 n only from knowledge 

of distributio n of X 1 A „ and from e„. One possible wa y to estimate e„ was 

established in iKovtun et al.l l|2005alrJ ): iKovtun et alJ 1 2005b|) suggested to use (i n 



as an estimator of /i. Conditions for consistency of this estimator are the main 
topic of the present article. 

Note that v, e„, /),„, etc. depend on the mixing measure /z. To stress such 
dependency, we use sometime notation u[/j], e n [/j], p, n [n], etc. We shall drop [/z] 
whenever the mixing measure ^ is obvious from the context. 
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We shall also use cr-algebras 3^ — a(X n ,X n +i,...) and the tail c-algebra 
X = n n 3n°- These u-algebras are considered as algebras of subsets of either A 
or = Q x A (in the latest case, they consist of cylinders built on subsets of A). 
It should be always clear from the context what case is considered. 

4. Main theorem 

The main theorem, proved in this section, establishes necessary and sufficient 
conditions for convergence jX n fi. It is formulated as nine equivalent conditions, 
which arc naturally splitted into three groups. The first group consists of conditions 
(JIJ and they are just statements of convergence. The second group consists of 
conditions J2J), Q, ©, an d ©; these conditions are expressed in terms of random 
variable G and related entities. The last group consists of conditions JJJ, JBJ), 
and ©. Condition (JZJ) is just statement of pairwise orthogonality of independent 
measures being mixed, while other two are criteria of orthogonality in terms of 
Hcllinger integrals. 

The condition JJJl, G = E(G | 3^°°) is-a,.s., looks surprising for the first glance: G 
as function of oj = (g,a) does not depend on a, while E(G | 3°°) does not depend 
on g. The only points where these two functions coincide is the graph of function 
eoo. But, according to condition 1(5} , this graph has f -measure 1, and thus equality 
holds z^-a.s. 

This condition has an important consequence: if the sequence of outcomes 
a = (ai,...) is generated by an independent distribution ? g , then eoo (a) with 
probability 1 gives this distribution. This fact justifies usage of conditional expec- 
tation E(G | X\ = ai , . . . , Xj = «j) as esti mation of individual position in phase 
space Q, suggested in lKovtun et al.1 l)2005bj) . 

Theorem 4.1. The following conditions are equivalent: 

(1) For every [i £ 7(Q), AnM t 1 

(2) For every £ 7{Q), Aoo[m] = M 

(3) For every /i £ 7{Q), ^[/i](reoo[/i]) = 1 (where Tf denotes the graph of a 
function f) 

(4) For every fi £ 7{Q), G = E(G | 3™>) v\jj.]-a.s. 

(5) For every fi £ 3 3 (Q), there exists a variant of regular conditional distribution 
of G with respect to 3°° (i.e., a function P(uj,B), uj £ £1, B £ "&(ff)) such 
that for every lu £ Q, the distribution P(uj,-) is concentrated at a single 
point. 

(6) For every fi £ CP(Q) ; there exists a set Q± C Q with fi(Qi) = 1 such that 
tail a-algebra X is sufficient for the family of measures {P g \ g £ Qi}. 

(7) V.g', g" , Q : //' •' g" > P g > ± P, 

(8) V</, g" £Q:g'^g" U°° =1 VPW) - 

(9) Vg',g" £Q:g'^g" =* ££ x Efii {VMg 7 ) ~ VW 7 ))" = °o 

Proof. In order to make the proof of main theorem more clear, we moved proofs of 
all technical facts to the next section. 

CJ ©• Follows from the proposition 13. 51 

© => Q). Take arbitrary g',g" £ Q and take fi £ T(Q) being concentrated at 
points g' , g" with weights a and 1 — a, where < a < 1. Then for every B £ 23 (Q), 
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n(B) = p,oo(B) = P /J (e 00 1 (i3)). In particular, for A' = e 00 1 ({sf'}), one obtains 
P ti (A')=K{9'})=a. 
Further, one obtains: 



g'-a = g- / P^da) = / g'P fl (da) = / e^a) P^(do 
J A' Ja> Ja> 



E(G | J 00 )^) v{du) = / G{lo) v(duj) ( =' 

QxA' JQxA' 

g' ■ u{{g'} x A 1 ) + g" ■ u{{g"} x A') C = ) «/ ■ aP^A') + 5 " ■ (1 - a)P g „(A') 

(equality (*) holds as G(ui) is a simple function, taking with non-zero probability 
only values g' and g"; equality (**) follows from definition of v, equation 10}). 

As g' and g" are linearly independent vectors, the above equation implies that 
P g '(A') = 1 and P g "(A') = 0, which means P g > _L P 9 », q.e.d. 

0« 0. Follows from lemmas and HT^Sl 

(J2J => (0). We shall show a stronger fact, that there exists a variant of regular 
conditional distribution with respect to the tail cr-algebra X, which has the required 
property and which serves as regular conditional distribution w.r.t. 'J 00 as well. 
Take P((g, a), B) = 5 ip ^ a - ) (B), where <p defined by lemma l5.25l For every oj = (g, a), 
P(ui,B) is, by definition, a probabilistic measure on (QjI^Q)), and this measure 
is concentrated at a single point. Thus, one has to show that for every B E 23 (Q), 
i/(G- x (S)|X)(a;) = E(/ G -i (B) |X)(w) = P(u,B) u-a.s. As by lemma QUI P(u, B) 
is X-measurable, one needs to show that for every Q x A G X 



(5) / I G ~i {B) (uj)p(du)= P(u,B)v(du) 

JQxA JQxA 

Using Robbins' theorem fsection IQj) . the left-hand side is reduced to: 
^G-i(B)M v(dm) = I I B (g)v(duj) = 

xA JQxA 



I B (g)P g (da) ) (i(dg) = / I B (g) ( / I A (a)P g (da) 1 p(dg) = 

I B (g)P g (A)fi(dg) 



where h is a homomorphism defined in section !?). 41 The right-hand side, in turn, is 
reduced to: 

P(uj, B) V (dio) = f 5 v[a) (B) v(dij) = 

<A JQxA 



S v(a) (B) P g (da) ) f,(dg) = I 8 g (B)P g (A)v(dg) = 

I B (g)P g (A)n(dg) 



Q 

which proves the equality ©. Finally, note that the proof of the equality (JjJJ 
remain valid, if A is taken from 5F°°. Thus, P(u>,B) is also a regular conditional 
distribution w.r.t. 3 ro ° . 
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© 0- Let P(ui, B) be a regular conditional distribution of G w.r.t. 3^°° , such 
that for every u> the distribution P(u>, •) is concentrated at a single point. Note that 
P(u>i B) as function of uo = (g, a) does not depend on g (as J 00 consists of cylinders 
Q x A); this allows us to write P(a,B). Le t further q(a) b e the point, which 
the distribution P(a,-) is concentrated at. By IShirvaevI l|2004l II. 7. 7), conditional 
expectation can be calculated using regular conditional distribution: 

E(G\?°°)(g',a , )= [ gP{a! ,dg) = g(a') 
Jq 

For arbitrary C £ 23(f2) one has v{C) = J A P(a,C a )P fJi (da), where C a — {g \ 
(g,a) € C}. Applying this formula to C* = {lu \ G(uj) = E(G|X)(w)} = {(g,a) \ 
g = g(a)} one obtains 

u(C )= [ P(a,{g(a)})P^(da) = [ l-P M (da) = l 

J A J A 

which proves that G(ui) — E(G|X)(o;) u-a.s. 
10} =>■ ©. Follows from: 

Teoo = {(g, a)\g = e^a)} = {(g, a) | G(g, a) = E(G\X)(g, a)} 

© ©• First, note that for every B £ "B(Q) one has (Bxi)fl Te^ = 
(Q x e^(B)) n r eoo . Then (as P{A) = 1 => P(S) = P(B n A) and ^ and 
P M are marginals of v) one obtains /i(-B) = v{B x A) = v{{B x A) H Teoo) and 
Aoo(S) = P M (e- 1 (B)) = j/(Q x e" 1 ^)) = i/((Q x e^B)) n Te^), which gives 
/«(£) = fica(B) for every Borel B, q.e.d. 

^ ©. Bv IShirvaevI (|2004L III.9.3, theorem 3), P g > _L P 9 » is equivalent to 
H(P g >, P g ") = 0, where H(P, P') is a Hellinger integral of order h. When random 
variables X\, . . . are independent (which is the case for distributions P g > and P g »), 
their joint distribution is the product of individual distributions, and Hellinger 
integral for the joint distributions is the product of Hellinger integrals for the indi- 
vidual distributions l|Snirvaevl Etui III. 9. 2). The direct computation (see section 
EH) gives H(P g ,, P g „) = Hf =1 Y%U y/Pjiig'Wig"), q-e-d. 

© ©■ Follows from lemma ICTI □ 

5. AUXILARY LEMMAS 

5.1. Hellinger integrals and related series. Let P^ denote the distribution of 
Xj that corresponds to g. P J g is defined on a finite probabilistic space {1, . . . , Lj}, 
and is fully determined by vector (flji(g), ■ ■ ■ ,PjLj (<?))• The Hellinger integral of 
order § (see IShirvaevI 12004 III.9.3) for P j g , and P j g „ is: 

Ll i 

(6) h(p 9 „ f* ,) = £ puig'Wig") 

i=i 

As random variables are mutually independent in distribution P g , the 

measure P g is a product of measures P^; thus, the Hellinger integral for distributions 
P g > and P g " is 

oo oo Lj 

(7) ^(p,,, p 9 ») = n H(p j gl) pi„) = n e y/w)w) 
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The important properties of the Hcllingcr integral arc: 
(8) (a) P' _L P" H(P', P") = (b) P' ~ P" => H{P', P") > 
We are also interested in related quantity 

oo Lj . N 2 

(Q) h + {p cjI , p g „) = E E ( yfM) - \fW 

j=i i=i ^ 

To simplify notation, we shall use 
(10) H(g',g") ^ H{P g ,,P a „); H+(g',g") d = f H+(P g ,,P g „) 

The equality of the Hellinger integral to zero may come from two sources: first, 
one of the factors in the product equals to 0, and second, all factors are positive 
but converge to zero sufficiently quickly. 

In the second case, one can employ the following considerations. For arbitrary 
sequence {x n } n with < x n < 1, one has 



(11) ]^[.t„ = y^lnfon) = -oo In — = oo 44> y^(l — x n ) 



oo 



Remark 5.1. If in ljTT|) allowed to be zero, all equivalences except the last 

one remain true; for the last step, only implication from right to left holds. 

Further, for every j 



E M) + E M') - 2 E \AW)/W) = 

I I I 

2 (± - E \fww^j 

Thus, one obtains: 

Lemma 5.2. If for every j , P J g , ~ P J gl , , then 

(12) H(g',g") = & H+(g',g") =oc 

In the first case, however, the equivalence (|12f) is not necessarily true. If, for 
example, fi{g') — (1,0, |, i, . . . ) and f3(g") = (0, 1, |, |, . . .) (all random variables 
Xj are assumed binary), one has 



H( 9 ', g ") = (VTo + VcTT) •^y|-| + vI"I)"""" = " 1 

while the right-hand series in l|12[l equals to 

(vT- Vo) 2 + (Vo-v 7 ! 



2 / _ _\ 2\ 

J. / 1 \ , / fl fl 



2 V 2 / ' \ V 2 V 2 

= 2 + 0+--- = 2 



Nevertheless, the equality to zero of H(g',g") for every pair g',g" is equivalent 
to divergence H + (g',g") for every pair g',g". We shall establish this weaker (but 
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sufficient for our purposes) fact in the lemma 15*1)1 below: but before this we need a 
couple of additional notions and lemmas. 

For g',g" S Q, let [g',g"] denote a closed interval with endpoints g' and g" (i.e., 
\g',g") = {(1 - t) ■ g' + t ■ g" | Q < t < 1}). Similarly, {g',g") denotes an open 
interval and (g',g") denotes the whole line passing through g' and g" . Note that 
Q is convex, bounded and closed; thus, for every g',g" G Q : [g',g"] C Q and 
intersection (g',g") (1 Q is a closed interval. 

Lemma 5.3. IfH + (g',g") = oo and [g',g"] C i/ien H+(g',g") = 00. 



Proof. Due to linearity of mapping /?, one has for every \ \/(3ji(g') — \J 'Pji(g")\ < 

\VW)-VW)\- □ 

Lemma 5.4. For every a, 6 € [0, 1] one /ias 



Proof. If 6 = 0, the inequality is verified directly. If 6 > 0, one can divide both 
sides by b and consider a function 



*)-(|-V5).(Vi- 1 ) , -^i±i- 1 ) 

By analyzing this function, one obtains that it has a single maximum on (0, 00) at 
point 1, which equals 0. □ 

Corollary 5.5. H+(g',g") = oc => H+{^-, g") = H+(g', = 00. 

Lemma 5.6. If for some g', g" £ Q one has H + (g' , g") — 00, then H + (g' , g") = 00 
for every g',g" € {§',§") D Q. 

Proof. If [g',g"] C [g',g"], the statement follows from lemma Otherwise, by 
taking g' {l) = g', g'fo = g" and either g[ i+1) = g' {i) , g'( i+1) = g< "^ g( " or g[ i+1) = 

g' -\-g"- 

9(1) 2 g( '' : 9"i+i) = 9'liy one comes to g' {n) , g'^ such that [g[ ny gJJ\ C Then 
the required statement follows from corollary 15 . 51 and lemma "5"^"1 □ 

Lemma 5.7. If g',g" 6 Q, f/ien /or every g',g" € (<?',<?") one has P 3 g , ~ P^„ /or 
every j . 

Proof. The only reason for PL 7^ P^„ is that for some I, (3ji(g') = and f3ji(g") > 
(or vice versa). But due to linearity of mapping (3 this may happen only for 
endpoints of the interval (g',g") (1Q. □ 

Corollary 5.8. Letg',g"eQ. Then H(g' , g") = H+(g' ,g") = 00 for every 

g',g" eCT,5"). 

Lemma 5.9. V ff ',.g" S Q : H{g',g") = <^ V.g',.g" G Q : H+(g',g") = 00. 

Proof. The implication from right to left follows from remark f5. II To prove impli- 
cation from left to right, assume premise and take arbitrary g 1 ,g" € Q. Further, 
take arbitrary g',g" S (g',g")- By assumption, H(g',g") — 0, and by corollary 15. 81 
H + (g',g") = 00. Then by lemmal5l)lH + fa'. a") = 00, q.e.d. □ 
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5.2. Tail space. In this and subsequent subsections we consider random vari- 
ables Xj as denned on measurable space (A,H(A)); thus, the er-algebras 5F™ = 
a(X x , . . . , X n ), = <r(U„ 3 rn ), 9% = a{X n ,X n+u . . . ), and X = fl„ ^) are 
subalgebras of 23 (./I). 

Consider a relation ~ on .A defined as: 

/-i o\ III def — i »/ III 

{13) a = dnv m > n : a m = a m 

Obviously, this relation is an equivalence relation; let a denote a coset of a G A, 
and A denote a factorspace of A. 

Unfortunately, the factor topology of A is trivial (a closure of every one-point set 
is the whole space) , and thus does not possess any useful for our purpose property. 

One can easily verify that for every A G X the implication a G A => a C A 
holds, and that for every a G A the coset a belongs to the tail a-algebra X. This 
makes the tail er-algebra X isomorphic to its image X = {A C A \ 3 A G X : 
A = {a | a G A}} under factor mapping. Any measure P on (A, 23(A)) can be 
restricted to er-algebra X; this restriction gives rise to measure P on (A,X). Also, 
any X-measurablc function / on A corresponds to a X-measurable function f on A 
defined as /(a) = f(a). 

The measurable space (A, X) is called a tail space. The slight advantage of 
considering the tail space instead of (A, X) is that atoms of X (in the sense of 
Boolean algebra) are one-point sets, while atoms of X are cosets of relation ~. 

In the subsequent subsections, we drop the ~ sign and use A, X, etc. to denote 
A, X, etc. This should not led to confusion, as any statement regarding (A, X) 
can be reformulated as a statement regarding (A,X), and in the most cases such 
reformulation consists just of dropping ~ sign. 

5.3. Orthogonality of measures restricted to X. For every measures P', P" 
on (A,T>(A)) the absolute continuity P' <C P" implies the absolute continuity of 
measures restricted to X, P'|x P"|x- We shall show that a similar implication 
holds for orthogonality in the case of family of measures {P g } g eQ- Namely: 

Lemma 5.10. Suppose that \/g',g" G Q : g' ^ g" =*> P g > _L P s ». Then ^g',g" G 
Q : g' g" =>■ P 3 '|x -L Pg"|x 7 i.e. there exist A G X such that P g i(A) — 1 and 

p s »(^) = Q- 

Proof. Assume premise of the lemma. By lemma 

Ef=i E ; (« - vW 7 )) 2 

diverges for every g' ^ g" . This implies that for every n the sum 

also diverges. From this, by applying considerations of subsection 15. II to the space 
A {n) = H™ =n {l, . . . , Lj}, one obtains P^, l) _L P^, } for arbitrary g' ^ g" (here P ( g n) 
denotes a measure on A^ defined in the way similar to the definition of P g on 
A). But P g ^ and P^, are marginals of P g i and P g n, respectively; thus, for every n 
there exists a set A n G 1^ such that P g /(A n ) = 1 and P fl "(A„) = 0. 

Now take A — H^Li U'jLn A n - It is easy to verify that A £ X and P g '(A) = 1, 
P g " (A) = 0, which proves the lemma. □ 
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Remark 5.11. The condition that the orthogonality takes place for every pair 
g',g" G Q is important. To obtain a counterexample, consider the case: Q = [0, 1], 
K = 2, all variables Xi, . . . are binary, A 1 = (1, 0, |, \, . . . ), A 2 = (0, 1, |, 5, . . . ), 
and /3(g) = gX 1 + (1 — g)X 2 . Then P _L Pi, but P |x ~ Pi|x- In this case, however, 

Pl~P3. 

4 4 

5.4. Pairwise orthogonality implies orthogonality in aggregate. In this sub- 
section, all measures P g , P M are considered as being defined on the measurable space 
(A, X) (see subsection 15.211 . We shall, however, drop the ~ sign in subsequent no- 
tation. 

We consider a mapping h : X — > "B(Q) defined as: 

(14) hA = {geQ\ P g (A) = l} 

According to Kolmogorov's zero-one law, P g takes only values and 1; thus, 
g £ hA is equivalent to P g (A) — 0. The fact that hA is a Borel set follows 
from proposition 13.41 as h A is the inverse image of Borel set {1} under mapping 
g^P g {A). 

Our first goal in this subsection is to show that h is (mod 0) epimorphism of 
cr-algebras. 

Lemma 5.12. Mapping h is a homomorphism of a -algebras. 

Proof. All required relations h(A'r\A") = hA'nhA", etc. and h(f] n A n ) = f|„ hA n , 
etc. can be verified directly. For example: 

MD„ A n ) = {g€Q\ P g (f) n An) = 1} = {g e Q \ VnP g (A n ) = 1} = 

n n {9eQ\P g (A n ) = l} = r\ n hA n □ 

A proof that h is "onto" (mod 0) requires establishing additional facts. 

We shall extensively use the following theorem, proved in iRobbinsI l|l948l Theo- 
rem 2; the wording below is adopted for our case): 

Robbins' Theorem. Let f : A — > R be an % -measurable nonnegative function. 
Then the function f(g) = J„ f(a) P g (da) is ¥>(Q) -measurable and for every mea- 
sure ix on Q 

(15) f f(a)P^da)= [ f{g)ix(dg)= [ ([ f(a)P g (da)) Lx(dg) 

J A JQ JQ \JA J 

The first important corollary of the Robbins' theorem is: 
Lemma 5.13. For every measure fx on Q and for every A S X, P fl (A) = (x(hA). 
Proof. 

P U (A) = JjA{a) P^(da) = J Qf -k(a) P g {da)) (x(dg) = 

f P g (A)ix(dg)= f I hA (g) Li(dg) = fx(hA) □ 

JQ JQ 

Lemma 5.14. V,/i" € V(Q) ■ p! < \x" => P w < P u „ . 

Proof. P W >{A) = 0o fx"(hA) = => fjf(hA) = & P^(A) = 0. □ 
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Lemma 5.15. Let f/,(J," G 3 3 (<5) be such that P^ <C P^". Then there exists a 
measure p" G 3 3 (Q) such that p" <C /i" and Ppn = P ^ . 



Proof. Let 



*<«) = §£(.> 



be a Radon- Nikodym derivative of P^' with respect to P M " . Let us fix any variant of 
z(a). By Robbins' theorem, the function y(g) = J A z(a) P g (da) is S(<3)-measurable 
and 

y(g) f x"(dg) = f ( f z(a)P g (da)) p"{dg) = [ z(o) P„«(da) = 1 
Jq \Ja / J A 

Thus, the set function p"{B) = J B y(g) fi"(dg) is a probabilistic measure on Q 
and p" <C p" . Further, using lemma 157131 and Robbins' theorem, one obtains for 
arbitrary A G X: 



Pp«(A)=/j"(ftA)= / I hA (g)y(g)Li"(dg) = 

IhA(g) [J A z ( a ) p 9 ( dfl )) = y f^JjA{a)z{a) P g (da)J p"{dg) 



I A (a)z(a) P^(da) = P^(A) 

A 



The intermediate equality (*) is verified directly: 

P g (A) = 1 =► I hA {g) = 1 and f z(a) P g (da) = [ z(a) P g (da) 

J A J A 



and 



P g (A)=0 => I hA (g)=0&nd / z(a) P g (da) = 



Thus, one obtains P^n = P /, q.e.d. We did not prove that different variants of 
z(a) lead to the same measure p" , but it is insignificant for this lemma, as it claims 
only existence of measure /I" with desired properties, and any variant of z(a) gives 
such example. □ 

Lemma 5.16. Suppose thatWg' , g" £ Q : j' ^ g" =>• P ff / _L P g </. Then continuity 
of /i G CP(Q) implies continuity o/P M . 

Proof. Assume, in contrary, that Ao 6 X is an atom of P^, i.e. P fJi (Ao) > and for 
every A G X, A C A , either P„(A) = P M (A)) or P M (A) = 0. 

Let Bo = hAo. Then /i(i?o) > 0, and we can consider a measure po = ju(-|-Bo)- 
It is easy to see that P M0 is carried by A and j4o is an atom of P M0 . 

Consider any measure // <C /io- By lemma 15 .141 P^/ <C P M0 . Thus, there exists 
a Radon-Nikodym derivative z(a) = dP^i/dP^. But as is atom of P Mo , z(a) is 
P Mo -a.s. a constant on Ao, and it follows from normalization conditions that this 
constant is 1. Thus, P M < = P Mo for every measure \i! <C Ho- 

Now return to the set Bo. As fio{Bo) — 1 and /xo has no atoms, it is possible to 
construct two sequences of subsets of Bo, {B' n } n and {B"}„, such that B' n JJ. {5'}, 
11 {5"}, 5' 7^ g", and /io(S;) > 0, /io(S;') > for all n. 
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Consider measures fi' n — (j,o('\B' n ) and /i" = /io(-|-B")- One has fi' n — > 5 g i and 
Jo" , whi le P^' = P^" = P Mo for all n. From continuity of the mixing operator 
l Kovturll20f)l . one obtains 

P g , = lim P- = lim P^„ = Pg„ 

n — *oo n — >oo 

which contradicts to the assumption of the lemma that P g > _L P g u . □ 

Lemma 5.17. Suppose that Vg',g" £ Q : g' ^ g" P g > _L P g ». Let p G 3>(Q) be 
a continuous measure. Then P M _L P g for every g £ Q. 

Proof. Assume premise of the lemma and take arbitrary continuous [i and arbitrary 
g. Let fx — + S g ). Due to linearity of the mixing operator, P^ = i(P M + P g ), 
and thus P M < P p and P 5 < P p . Let 

dP dP 

*(a) = 5 p^(o), z ( a ) = ^( fl )' A = {a | z(a) • z(a) > 0} 

Then P^ _L P g is equivalent to Pp i (Ao) = 0. 

Assume in contrary that Pp i (Ao) > 0. Take arbitrary A\ C Ao. If < Pp_{A\) < 
Pp.(A ), then P 9 (Ai) > and P g (A a \ Ax) > 0, which contradicts to the Kol- 
mogorov's zero-or-one law. Thus, A$ is an atom of P^, and consequently an atom 
of P M , which contradicts to the lemma 15.161 □ 

Lemma 5.18. Suppose that Vg',g" € Q : g 1 ^ g" => P g > _L P g „. Let p e 7>(Q) be 
a continuous measure, and fif S 3 (Q) be a counting measure. Then P M _L P M /. 

Proof. Any counting measure can be represented in form p! = X)j=i a i$gn where 
ai > and J^Oj = 1. Then due to linearity of the mixing operator P ^ — ^ aiP gi . 

For every i — 1, . . . take such that P ^{Ai) = 1 and P 9i (Ai) = (such sets exist 
due to lemma 1537) . Now take A = f|i ^. Then P M (A) = 1 and P^(A) = 0. □ 

Lemma 5.19. Suppose that Vg',g" e Q : g' ^ g" => P g < _L P g ". Let p e foe 

a continuous measure, F be a closed set such that n(F) > 0, // = and z(a) 

be a Radon- Nikodym derivative ofP^' with respect to P^. Let Aq = {a \ z{a) > 0}. 
Then p(F A hA ) = 0. 

Proof. As fj,'(hAo) = P^^Aq) = 1, one obtains fjf(F\ hAo) — 0; consequently, 
fi(F\hA Q ) = 0. 

Next, assume in contrary that p(hAo \ F) > 0. As F is closed, one can construct 
a sequence of sets {B n } n such that B" C /jA \ F, (J,(B n ) > 0, and B n JJ. {g} for 
some g € /iA \F. Let Li n = p(-\B n ). Then /i n <C li, and by lemma IS~FH P,, <C P M . 
Let 

dP 

It is always possible to choose such variants of z n (a) that A n C Aq, A n+ \ C A n . 
Let = hA n n F. Then C B' n and p'(B' n ) = Li'(hA n ) = P^{A n ) > (as 

Pp>(A n ) = e> P„(A n ) = 0). 

Let fi' n = ^'(-|_B^). We claim that P M ^ <C P M „. To prove this, take any set A 
such that Pfi> n (A) > 0; then P^(A) = (7 n (hA) = Li' n (hAn hA n ) = P^fin A n ); 
further, as [i' n ^ p! <C P^(A n A„) > 0, and consequently P M „(^4 n A n ) > 0, 
q.e.d. 

Further, we claim that there exists a point g' £ F such that Ve > Vn : 
li' n (U e g') > 0. To prove this, suppose the contrary, i.e. that for every point g € F 
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there exists e(g) such that f/ n (U s ( g \g) = for all n starting some n(g) (note that 
for n' > n one has fjf n , <C (J n , and thus n' n {U) = L*f n <(U) = 0). This system 
covers F, and as F is compact, it contains a finite subcover {E4(g;)<7i}™ i- Let 
n = max(ra(gi), . . . ,n(g m )). Then 1 = fi' no (F) < £\ ^n Q { U e{ gi )9i) = 0. This 
contradiction proves the claim. 

Now take B',[ = B' n n (7 1/n5 ' U {<?'} and < = m'„("I^)- One has B' 7 [ |j {g'}. 
As <C /x^j, one obtains P p » -C P^ <C P M „. By lemma l5~15l one can construct 
measures //„, <C M t ?, such that P ^ = P^n . Finally, using continuity of the mixing 
operator l|Kovtunl 12005). one obtains 

P g . = lim P^u = lim P- = p 

n — >oo n — >oo 

which contradicts to the assumption of the lemma that P g > 1 P s . □ 

Lemma 5.20. Suppose that Vg', g" G Q : g' ^ g" =>■ P g < -L Pg" ■ TTiera /or every 
continuous measure /i G 3 3 (Q) and for every B G 23(Q) f/iere exists A G X such that 
^{B A = 0. 

Proo/. Take arbitrary continuous fj, G Let S = {B e 23(Q) | 3 A e X : 

/i(B A = 0}. By lemma 15.191 all closed subsets of Q belong to Ho- As 
h is a homomorphism of cr-algebras (lemma I5.12JI . 23o is closed under finite and 
countable unions and intersections and under complements. Thus, 23o coincides 
with S(Q). □ 

Lemma 5.21. Suppose thatVg' , g" G Q : .<?' 7^ .°" =^ Pg' -L Pg" ■ Then for every 
counting measure fi G 5 (Q) and /or every B G 23 (Q) t/iere exists A G X suc/i i/iai 
n(BAhA) = 0. 

Proof. Let /i = 532=1 °i^gi: where Oj > and 53 a i = L Then P p = 53 a iPg ; - 

For every n ^ m take A nm such that P gn (A nm ) = 1 and P gm (A nm ) = 0, and 

take An = flm^n Then P 9n (A n ) = 1 and for every m ^ n P gm (A n ) = 0. 

Now for every B G 23(Q) take Ab = V}{A n \ g n G B}. It is easy to verify that 

fj,(B A hA B ) = 0. □ 

Lemma 5.22. Suppose that Vg',g" G Q : g' ^ g" P g > _L P s ». Then for 

every measure /x G T(<5) and for every B G 23 (Q) i/iere exists A G X suc/i £/ia£ 
u(B A = 0. 

Proof. Every measure u on Q can be represented as a/i' + (1 — a)/j," , where // is 
continuous and u" is counting. 

By lemma l5~T51 one can find sets A', A" G X such that P^ (A') = 1, P^» (A") = 1, 
and A' n A" = 0. 

Now take arbitrary _B G 23 (Q). By lemma IB. 201 there exists A\ G X such that 
fi'(BAhAi) = 0, and by lemma l5~2"Tl there exists A 2 G X such that (i"(BAhA 2 ) = 
0. It is always possible to choose these sets such that Ax C A' and A2 C A". Thus, 
Ax n A 2 = and fJ,'(hA 2 ) = li"{hA x ) = 0. 

Finally, 

jti(B A /i(Ax U A 2 )) = A hA x A hA 2 ) = 

af/(B A hAx A hA 2 ) + (1 - a)n"{B A hA x A hA 2 ) = 

ap!{B A hAx) + (1 - a)n"{B A hA 2 ) = a- 0+(l-a)-0 = □ 

Lemma f5 . 221 allows us to obtain a corollary which is of interest by itself. Namely: 



18 



MIKHAIL KOVTUN, ANATOLIY YASHIN, AND IGOR AKUSHEVICH 



Lemma 5.23. Suppose that Vg',g" £ Q : g' ^ g" => P g / _L P ff ». Then V>i,/Z2 £ 
3>(Q) : fii -L fi 2 => P Ml -L P M2 . 

Proof. Assume premise of the lemma and take arbitrary fj,±, \x 2 such that /ii _L /i 2 . 
Then there exists Borel B l7 B 2 C Q such that /ii(Bi) — 1, fi 2 (B 2 ) = 1, and 
B 1 nB 2 = 0. 

Let /z = + /i2). By lemma 15.221 there exist sets A! X ,A! 2 £ X such that 

^(SiA/iAi) = n{B 2 /\hA' 2 ) = 0. As P M (A' 1 n J 4 2 ) = /ti(^in/iA' 2 ) = /i(SinS 2 ) = 0, 
the sets Ai = \ (A'j n A 2 ) and A 2 = A 2 \ n A' 2 ) also satisfy /u(Bi A hAi) = 
H(B 2 A /iA 2 ) =0. 

Now one has P fJ , 1 (Ai) = fii(hAi) = fii(Bi) = 1; similarly, P M2 (A 2 ) = 1. As A\ 
and A 2 are disjoint, one obtains orthogonality of P Ml and P M2 . □ 

On the other hand, lemma 15.221 can be easily derived from lemma 15.231 For 
this, take arbitrary /j, £ ?{Q) and B £ 25 (Q) . If /i(-B) = 1 or fi(B) = 0, one can 
take A = A or A = 0, respectively. Otherwise, one can consider measures n{-\B) 
and n(-\Q \ B). These measures are orthogonal, and by lemma 15. 231 measures 
P^f.is) and P m (.|q\s) are orthogonal as well. Take A such that P^.im^) = 1 and 
P ' fj,r.\Q\B){A) = 0. It is easy to see that [i{B A hA) = 0, which proves lemma 15.221 

Lemma 15.221 shows that for every /i £ 7(Q) the metric structure of spaces 
(Q,"B(Q), n) and (A, X P„) are isomorphi c. This allows us to employ Rokhlin's 
technique (see lR,okhlinLfT94alR.ohlinLFm5l to obtain the main result of the present 
subsection. 

As Q is a complete separable metric space, (Q, 23 M (<2), p) is a Lebesgue space 
for every Borel measure fj, (recall that 23 M (Q) denotes a Lebesgue completion of 
Borel a- algebra 23 (Q) with respect to measure /x, and p denotes the corresponding 
completion of measure //). 

Let us fix some measure \x on (Q,"B(Q)), and let (5 = {I^}?^ be a topology base 
of Q; it is also a Rokhlin's basis of (Q, 23 A1 (Q), p). Let further f 1 , be a coset of T, 
by ideal of p- negligible sets Ip,. 

By lemma !5'.22l there exists a set Ci £ X such that hCi £ i\. Let Aj = Ti A /iCi, 
and let Qo = Ui^i A», Qi — Q \ Qo- Then we have exact equalities hd = Ti n Qi 
and h(A \ d) = Qi\r 1 . 

Note that both Qo and Qi are Borel sets (as all Ti are open, and all hCi are 
Borel). 

For arbitrary g £ Q u let T.^g) = T.nQi if 3 € T h and T,( 5 ) = Qi^ if g T t . 
Correspondingly, let Di(G) = C t if g £ T tl and Di(g) = A\Ci if g £ Ti. Then 
hD i (g) = T l (g). 

Finally, let A g = f). D^g). As h is homomorphism of er-algebras, one obtains 

hA g = MfliACff)) = n^A(5) = fli T i(9) - This me ans that P ff (A s ) = 1 
and for every g' ^ g, P g i(A g ) = 0. Thus, we proved: 

Lemma 5.24. Suppose that Vg' ,g" £ Q ■ g' ^ g" => P s ' J- P 9 ". Then for every 
measure /i £ 3 3 (Q) there exists a Borel subset Qi Q Q of full measure, /j(Qi) = 1. 
and a mapping ip : Q\ — * X : g i— > A g satisfying (a) V g £ Q\ : P g (A g ) = 1 and (b) 
Vg,g' £Qi-.g^g' ^ A g nA g , =0. 

As sets {A g } g £Q 1 are disjoint, one can consider a mapping A — > Q : a i— > g a , 
defined as g> = g, if a G A g , and <7 a = 50 for a £ \J ge g 1 A g ; go £ Q is taken 
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arbitrarily. It is easy to see that the mapping a i— ► g a is 23(Q)-X- measurable. Thus, 
we have 

Lemma 5.25. Suppose that Vg',g" G Q ■ g' ^ g" => P s ' J- P 9 ". Then for every 
measure fi G J'(Q) i/iere exists a Borel subset Qi Q Q of full measure, /i(Qi) = 1, 
and a 23(Q)-X -measurable mapping ip : A — > Q : a i— ► g a satisfying Q\ C y(A) and 
PgCv'Hs)) = 1 f or ever V 9 G Qi. 

5.5. Conditional probabilities with respect to tail er-algebra. Let P be a 

probabilistic measure on (A, 13(A)). We say that P satisfies 0-1-law, if for every 
A e X either P(A) = 1 or P(A) = 0. 

Note that all measures P g , g G Q, satisfy 0-1-law. 

For measures satisfying 0-1-law, conditional probabilities with respect to tail 
er-algebra X can be expressed directly. Namely, a well-known fact is: 

Lemma 5.26. Let measure P satisfy 0-1-law. Then for every B G 'B(-A) one has 
P(B\X){a) = P(B) /or P-almost all a & A. 

With this we can easily prove: 

Lemma 5.27. Suppose that Vg',g" G Q : g' 7^ g" =>■ P 9 ' _L P ff ". TTien /or every 
measure fi G 3 5 (Q) i/iere exists a Borel set Q\ of full measure, n(Qi) — 1, such that 
tail a -algebra X is sufficient for the family {P g } g eQi ■ 

Proof. Take Q\ satisfying conditions of lemma 15. 251 For a G A and B G 23 (.A), let 
P(a, B) = P v ( a )(B) (where ip was defined in lemma l5*.25(l . By definition, P(a, B) is 
a probabilistic measure on (A, r B{A)) for every fixed a € A. Further, by proposition 
13.41 and lemma IB. 251 P(a,B) is X-measurable as function of a for every fixed B, 
and by lemma IB. 251 for every g G Qi and for every fixed B, P(a,B) — P g {B) 
for P s -almost all a. Finally, by lemma 15.261 P(a,B) = P g (B\X)(a) P s -a.s., which 
completes the proof. □ 

The inverse statement is also true: 

Lemma 5.28. Suppose that for every measure fi E T'iQ) there exists a Borel set 
Qi of full measure, (i(Qi) = 1, such that tail a-algebra X is sufficient for the family 
{P g } geQl ■ Then \fg', g" G Q : g' + g" => P g , ± P g „ . 

Proof. Take arbitrary g',g" G Q such that g' ^ g" and consider a measure /j, = 
\{5 g i + 5 g "). Then any set Q\ of full measure fi contains g' and g" . Sufficiency 
of X means that there exists a function P(a,B) such that P g /(-B|X)(a) = P(a,B) 
P 9 '-a.s. and P s "(B|X)(a) = P(a,B) P 9 »-a.s. Family {P g } g eQ consists of distinct 
measures; thus, there exists a set Bq E H(A) such that P g i(Bo) ^ P g t/(Bo). 

Take A' = {a\ P{a,B ) = P g >(B )} and A" = {a \ P{a,B ) = P g »(B )}. By 
construction, A' n A" = 0, and by lemma l5~2lfl Pg'(A') = P g »(A") = 1, which 
proves orthogonality of P g i and P g » . □ 

6. Discussion 

6.1. Orthogonality in aggregate: for "all" or "almost all" g G QI The 

result proven in lemma 15.241 can be reformulated as "pairwise orthogonality of 
measures {P g } g eQ implies orthogonality in aggregate for almost all measures from 
this family." The "almost all" clause of the above statement can be based on any 
measure fi on Q — note, however, that sets A g may depend on the choice of /i. 
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One can also see that there is no obvious modification of our proof, which gives 
orthogonality in aggregate for the whole family {P g } g eQ- 

A natural question arise: whether the absence of full orthogonality in aggregate 
is an immanent property or it is merely a deficiency of our proof? 

This question can be also formulated in terms of lemma 15. 221 which states that 
homomorphism h : X — > 23(Q) is "onto" (mod 0); again, "(mod 0)" holds for any 
measure /i on Q — and again, set A satisfying fi(B A A) — may depend on the 
choice of /i. This point of view can be (less formally) expressed as: is the tail 
CT-algebra X sufficiently rich to have a factor-algebra isomorphic to 23 (Q), or one 
can obtain only isomorphism (mod 0)? 

6.2. Structure of tail cr-algebras. Structure of tail cr-algebras deserved not too 
much attention. Such lack of interest can be explained, first, by the fact that tail cr- 
algebras are "trivial" from probabilistic point of view (any independent distribution, 
being restricted to the tail cr-algebra, takes only values and 1), and second, by the 
absence of topology or other "good" structure generating tail cr-algebras (cf., for 
example, a comprehensive study of Borel sets by means of topology and descriptive 
set theory). 

A study performed in the present paper demonstrates that an explicit investiga- 
tion of properties of tail cr-algebra might bring fruitful results: technical basis for our 
main theorem is existence of homomorphism h : X — > 23 (Q), which is epimorphism 
(mod 0), proved in lemma 15.221 An important corollary of the latest fact is that 
for every measure /i on (Q, 23(Q)) the metric structures of spaces (Q, 23(Q), ^) and 
(A, X, Pfj,) are isomorphic. 

Note that X is constructed from ranges of random variables X\ , . . . ; nothing from 
distribution of X\, ... is involved in construction of X. Lemma 15.221 establishes 
existence of homomorphism h for any set Q — provided that Q is compact and 
continuously parameterize pairwise orthogonal family of independent measures. In 
other words, X is universal with respect to family of cr-algebras {23(Q)}q. It looks 
like this universality is a characteristic property of tail cr-algebras, and might be a 
useful tool for further investigation of their properties. 

6.3. Connection with the strong law of large numbers. One fact that we 
have proved in this article is that pairwise orthogonality of considered family of in- 
dependent distributions implies their orthogonality in aggregate. Informally speak- 
ing, one can say the sets A g , provided by lemma 15.241 are "sufficiently small" to be 
disjoint. 

Another way to obtain "small" sets with P g measure 1 is provided by the strong 
law of large numbers. In some sense, the "smallness" of the set of outcomes provided 
by the strong law of large numbers is one of cornerstones of probability theory. 
When random variables Xj are identically distributed for every g, the sets given 
by the law of large numbers for different P g are disjoint. 

Consider, however, the following example. 

Let Xj be a sequence of binary random variables taking values and 1, Q = 
[-1,1], and for every g e Q, P g (Xj = 1) = 5 + 57- and P g (Xj = 0) = \-i£r- It is 
easy to verify that condition © of the main theorem is satisfied; thus, distributions 
P g are pairwise orthogonal. Now from lemma l5*.24l one obtains existence of a disjoint 
family of sets {A g } ge Q 1 (where Qi has Lebesgue measure 1, and thus has cardinality 
of continuum) such that P g (A g ) = 1. 
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The strong law of large numbers in form of Kolmogorov l|Sbirva,evlEnfii IV.3.2), 
adopted for our case, states that for every increasing unbounded sequence of positive 
reals {b n } n such that w < 00 ^ ne se ^ 

A g LN *> ={a€A \ Sl^z^fel) ^ 0} 

has P 9 probability 1. 

Using the facts that Y^t° pr < 00 implies that ^ — * and that 

n n n t 

E = !) - E p 9 '^ = !) = E 9 -^f = 

one obtains that Ag W ' = A^f^ for all 3,5' e Q. 

Thus, the strong law of large numbers may produce sets of probability 1 that 
can be made "significantly smaller" — namely, the set produced by the strong law 
of large numbers can be splitted into continuum of subsets, each of which has 
probability 1 with respect to corresponding distribution. 

We believe that further investigation of consequences of this fact might bring 
interesting results. 

6.4. Constructiveness of proofs. The proof of existence of set A g in lemma 
15.241 is non-constructive; moreover, it explicitly uses axiom of choice. It would be 
desirable to find a constructive description of sets A g (e.g., as in the strong law 
of large numbers), as it would simplify analysis of their properties. In particular, 
we expect that constructive description of sets A g would clarify the answer to the 
question of section 16.11 
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